IN THE UNITED STATES PATENT AND TRADEMARK OFFICE , , 

In re the application of 

Barry Katz Group Art Unit: 2615 ^5 jp^ j 

Serial-No.: 08/741,308 

Examiner: Amelia Au 

Filed: October 30, 1996 

For: ASYNCHRONOUS VIDEO EVENT 
AND TRANSACTION DATA 
MULTIPLEXING TECHNIQUE FOR 
SURVEILLANCE SYSTEMS 



To the Commissioner of Patents and Trademarks: 
Sir: 

DECLARATION TTNIW R 37 C.F.R. 1.131 

This declaration is to establish completion of the invention in this application in the 
United States, at a date prior to February 4, 1994, the effective date of the prior art 
patent that was cited by the examiner. This declaration is submitted prior to final 
rejection. 



I, BARRY KATZ, declare as follows: 



1 . I am the inventor of the invention disclosed in the above-identified 
patent application which is a continuation of Ser. No. 08/232,363 filed on April 25, 
1994. 

2. I completed my invention, successfully performed it and disclosed the 
same to others in this country prior to February 4, 1994 the filing date of the application 
from which the United States Patent No. 5,491,511 matured. 

3. Prior to February 4, 1994, I prepared a written description of my 
invention which is attached hereto as Exhibit A. 

4. Exhibit A describes means a) and b) of pending Claim 17 in sections 
D. through G.; means c)~ e) are described in sections H. through J. and means f) is 
described in section K of Exhibit A. 

5. My claimed invention is further described in an earlier written 
description attached hereto and marked Exhibit B. The original pages of the document 
identified as Exhibit B are unnumbered. The pages of Exhibit B have been numbered for 
convenience of reference. 

6. Means f) of Claim 17 is more fully described on the next to last page 
of Exhibit B. The reference on that page to "our first patent" is to U.S. Patent No. 
5,216,502 which is referred to in my present patent application at page 4, line 25. 

7. Prior to February 4, 1994, 1 assembled and tested a system which 
comprised the means described in Claim 17. This system was demonstrated to the 
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Pennsylvania Turnpike Authority, as described on page four of Exhibit B. The tapes 
from this demonstration have been erased and re-used! 

8. I do not know and do not believe that the invention has been in public 
use or on sale in this country or patented or described in a printed publication in this or 
any foreign country for more than one (1) year prior to my application and I have never 
abandoned my invention. 

The undersigned declares further that all statement made herein of his own 
knowledge are true and that all statements made on information belief are believed to be 
true; and further these statement were made with the knowledge that willful false 
statements or the like so made are punishable by fine or imprisonment, or both, under 
Section 1001 of Title 18 of the United States Code and that such willful false statements 
may jeopardize the validity of the application or any patent issuing thereon. 



DATE: May 14, 1997 




Barry Katz 



Sole Inventor 
A US citizen residing at 
503 Cindy Circle 
Penllyn PA 19422 
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I. Claim a system that: 

A. Records images for duration of operations at all operation stations. 

1. Duration of operation defined as the time during which the activities of the enterprise 
or institution occur. 

2. Operation stations defined as the site of operation, that is the area where human and / or 
machine activities take place. For example: 

a) Cash registers. 

b) Toll payment area of Automatic Vehicle Identification based toll collection system. 

c) Ticketing station at airport 

d) Gas pump. 

e) Time and attendance clock. 

3. Images defined as reflected light from objects at the operation station. 

B. Records signals, data messages and data streams from instruments located at the operation 
site and / or associated with the operation site. 

1. Signals include those generated by : 

a) Physical security system components such as motion detectors, infrared detectors, 
contact closures etc. 

b) Communications signals such as those used by phone systems, data networks, and 
serial or parallel devices. 

2. Data messages : 

a) terminals, media readers such as magnetic card stripe readers and bar code readers. 

b) Peripheral components such as automatic vehicle classifiers ( AVC ) automatic 
vehicle identifiers ( AVI ), image extraction systems. 

3. Data streams : 

a) Data streams such as those between controllers and there associated components in 
POS systems, access control systems, toll payment systems. 

b) Data streams from devices connected to components that lack communications 
capability to collect data from them and communicate it to devices capable of 
storing and processing the data. 

C. Processes these signals, data messages and data streams, to create: 
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C. Processes these signals, data messages and data streams, to create: 

1. Event messages. 

2. Transaction level messages. 

3. Update messages. 

4. Time stamp messages. 

5. Source identification messages. 

D. Stores signals, data messages, data streams, event and transaction level messages, as well as 
update, time stamp, and source identification messages to memory. At a minimum, to 
dynamic and static data storage media such as RAM, EAROM, magnetic disk. 

1. Signals, data messages, and data streams, etc., are to be stored in data structures that 
range in complexity and power from arrays to RDMS. 

2. All records containing signals, data messages, and data streams, etc., stored have a time 
stamp and source identification field 

E. Stores signals, data messages, data streams, event and transaction level messages, as well as 
update, time stamp, and source identification messages in encoded form in, or 
synchronously with, images on various media. 

F. Provides certification stamp on those images which are encoded with data as part of image 
and not separately as in the case of storing data in soundtrack. 

1 . Certification stamp could be encrypted code that requires lookup key to provide valid 
time, dale, source, and sequence data. 

2. Location of certification stamp could be encrypted and password protected. 

3. They ability to provide the certification supports the use of the image and its encoded 
data as evidence. 

G. Stores images and encoded data on various media. 

1. Video tape, magnetic disk, optical disk, etc. 

2. All images are stored with source and time stamp encoded. 

a) Source and time information is stored in the image or with the image as in the case 
where sound accompanies the image, in a manner that does not effect the clarity of 
the image. 

b) Other data and data elements, such as signals, data messages, data streams, event and 
transaction level messages, as well as update, time stamp, and source identification 
messages can also be stored with the image in a similar manner so as not to interfere 
with the clarity of the picture. 
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b) Other data and data elements, such as signals, data messages, data streams, event and 
transaction level messages, as well as update, time stamp, and source identification 
messages can also be stored with the image in a similar manner so as not to interfere 
with the clarity of the picture. 

c) Means are provided to assure that the picture, time stamp and the data are 
synchronously associated in time. 

H. Includes processes for developing montages and sequences of images and data for 
documenting and explaining, if possible, behavioral aspects of transactions and interactions 
between the people and devices, that occur at operation sites. 

1. Processes ( On Line Transaction Processing ), including programs that use : 

a) Incoming data. 

b) Previously recorded and stored data. 

c) Data stored in system setup tables. 

2. System setup tables that allow the user to define as notable exceptions: 

a) Events. 

b) Sequences of events. 

c) Transactions. 

d) Sequences of transactions, etc. 

I. .Includes processes that search, find, and verify images and sequences of images that arc 
synchronous with the event and transaction data included in the notable exceptions. 

1. Images are stored on various media, from RAM to holographic memory. 

2. Images include time stamp, source, and other event or transaction data that is encoded 



a) Time stamp, certification stamp, source and other data is used to verify that the 
image belongs in the montage that is to be assembled. 

J. Includes processes to assemble montages and sequences. 

1. These have explanatory power which consists of how well the edited combination of 
pictures and data explain the notable exception. 

a) Notable exceptions can be explained by a number of alternative explanations. 

b) Notable exceptions raise questions. 



in them. 
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(1) These questions can be answered by reviewing pictures and data synchronously 
recorded at the operation site. 

(2) The appropriate explanation can be found It may include a known alternative or 
it may not. 

(3) If a new explanation is required, it can be added to the appropriate setup table. 
K. Includes processes to output assembled montages and sequences to various devices. 

1. Communications devices. 

2. Memory devices. 

3. Display devices. 

4. Processing devices. 
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I. VTDM applications in store and forward environments: 



A. For VTDM applications there are two store and forward architecture's. 



1. Systems that send all of the transaction data by the conclusion of the 
transaction. 

a) We have already discussed these applications in reference to the patent. The Tower 
Records system represents a solution for systems where completed transaction data is 
transmitted at the end of the transaction. 

b) We called the solution the "screen pause feature". 

(1) In the Tower Records system, the system sends transaction data in 
128 byte packets. The packet transmission begins with the start of 
the transaction and is completed at the end of the transaction when 
the tender key is hit. 

(2) We provide an application that allows the user to freeze the data 
display in the overlay while the video tape is rewound to the 
beginning of the transaction. 

(3) The user can then review the transaction while looking at the 
transaction data in the overlay. 

(4) Line markers are provided on the overlay to enable the user to 
move a pointer on the overlay to each item as it is registered by the 
clerk in the picture. 

(5) The system encodes the next transaction serial number on the 
sound track of the video tape at the end of each transaction. 

(6) The user can use this serial number to be certain that the 
transaction behavior that he is seeing corresponds to the 
transaction data in the data overlay. 

c) This solution works well for systems that provide completed transaction transmission 
by the end of the transaction. 
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B. There are systems where transaction data is stored beyond the 

transaction termination time, where several transactions are buffered 
before transmission to the host. 



1. In these cases up until now we have not been able to provide VTDM 
applications. 

2. Systems where transaction and event data are stored and not 
forwarded until well after the end of a transaction are common. 

a) One example are the toll systems at the Delaware River Joint Toll Bridge Authority 
and The Pennsylvania Turnpike. 

(1) These systems capture toll transaction data in a lane controller. 
The information comes from a toll terminal, card reader, loop 
detectors, treadles, indicator lights etc. located in each toll lane. 

(2) The data from each lane is stored in the lane controller and sent to 
the plaza controller at irregular intervals. 

(3) Several transactions take place between data transmissions. 

(a) In toll transactions, the vehicle exits the lane and the video scene 
at the end of the transaction. If several transactions occur before 
the data is transmitted, there is no way to link the data with the 
behavior other than the data freeze or data pause method described 
above. 

(b) The data pause application is not acceptable to toll authorities. It 
requires too much time to review tapes and does not work well with 
automated tape editing. 

II. During the past year DBS has designed proprietary circuits to perform the functions 
needed to record data and pictures 
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A. The AM90 is a proprietary circuit developed by DBS which inputs data 
and encodes it and outputs it to the video tape sound track, vertical 
interval, or within the visible video. 



III. To remedy this situation and to provide a VTDM for applications with store and 
forward data transmission we recommend the following: 



A. The VTDM computer encodes the video tape with facility, camera 
identification, date and time messages. The time messages can be 
inserted every 100 milliseconds. 



B.The VTDM simultaneously captures the transaction data from the lane 
controller. These messages include several transactions. Each 
transaction includes time for that transaction. Each transaction 
message can be stored in a file with the date, lane, terminal id, etc., 
and the time. 



1. The file would contain a list of messages indexed by date, time, lane, 
etc. 

2. Upon replay, the user selects the target lane. 

3. When the video tape is replayed, the VTDM inputs the date and time 
messages. 

a) The VTDM acquires the time messages from the tape and looks up the messages in 
the data file on the disk for that time and the target lane. 

b) The VTDM then displays the messages from the targeted lane in the data overlay. 

4. At the Pa. Turnpike, the lane controller stores three transactions and 
forwards all three at the end of the third transaction. 
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a) To provide VTDM applications for the Pa. Turnpike we would apply the solution 
described above. 

(1) We place one camera at the plaza to view four to six lanes from the 
exit side. 

(2) We install a VHS VCR for each camera. 

(3) We connect the VTDM computer to the lane controller 
communications so that the VTDM computer receives all of the data 
messages that are communicated between the lane controller and 
the plaza computer. 

(a) We encode the VHS VCR tape with data messages that identify the 
plaza, and the date. 

(b) We encode time messages every 100 milliseconds. 

(c) We store to the disk every transaction message with the plaza 
number, lane number, date and time. 

(4) When we replay the tape, we indicate which lane we want to target 
for data to be displayed in the overlay. 

(5) When the tape is replayed, the time, date plaza id etc. data 
recorded on the sound track or in the video, are read into the 
VTDM. 

(6) The VTDM reads the time and searches the data file for messages 
that have the targeted lane identifier and the matching time and 
date. 

(7) When the messages are located, they are formatted and displayed 
in the data overlay. 

(8) The time of the behavior in the picture now matches the data 
displayed in the overlay. 

IV. Other applications include: 
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A. Credit card validation. 



1- Many businesses have CCTV systems for register surveillance. These 
systems typically use VCRs for recording. 



2. Typically credit card validation is performed at the register site. 



3. An AM 90 board would make it possible to encode the video tape with 
time and location messages. 

a) The data from the card validation transaction has to be written to disk by the register 
system of collected by a computer and then transferred to a disk 

b) Upon replay of the video tape, the VTDM will input the time and location messages 
from the video tape and us them to search for the messages from the validation 
transaction disk. 



B. Bar code reading systems. Guard Tour Example. 



1. Guard tours systems traditionally have consisted of a portable time 
clock with an input for coded keys and a printed paper output. 

a) The coded keys were installed at locations around the area protected by the tour. 

b) The keys were frequently chained to the wall. 

c) Each key was individually coded to identify the location where it was installed. 

d) The clock added time and date to the event record. 

2. The modern electronic guard tour has the following advantages. 
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a) It produces machine readable data which can be communicated to a computer. 

b) It is less expensive to install bar codes that identify areas than to install the keys. 

c) With wireless communication you can have a real time guard tour which allows you 
to enunciate alarms for late or over due readings. 



3. There are two types of bar code data applications that can be used with 
VTDM. 

a) Applications where the bar code event and transaction data are transmitted via a 
wireless communication system. 

b) Applications where event and transaction data are stored internally in the bar code 
reader or stored on some removable medium such as a disk. 

4. In the wireless applications the bar code scans and the time and date 
are transmitted to a central controller in real time as they occur. 



5. We could use the existing VTDM technology to write the bar code and 
time data onto the video tape sound track. 



6. With the bar code guard tour applications where the scan events are 
stored in the portable reader, we would need to use our new store and 
forward VTDM application. 

a) Typically the scanner stores the data in its memory. 

b) The user connects the scanner to the host computer which inputs the scan event data 
into a file which is stored to the computers disk. This file is used for all of the guard 
tour reports and applications. 

c) If we synchronize the time in the scanner to the host computer time, and use the host 
computer time to set the clock on the AM90 we will have the same time for all three 
system components. 
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d) The AM90 will write the time and date every 100 milliseconds on the VCR sound 
track, vertical interval or in the visible video. 

e) After the scan event messages in the portable scanner have been uploaded to the host 
computer, a file indexed by time and bar code id will be written to a disk. 

f) The disk will be used to load the file into the VTDM memory. 

g) A video tape with pictures of the scenes that include the guard tour bar code reading 
stations and the encoded time and date on the sound track will be replayed on the 
VTDM VCR. 

h) The user will enter in the scanner id into the VTDM to identify which scanner 
messages are to be filtered out, formatted and displayed in the data block. 

i) The VTDM will then read in the time and date messages from the VCR tape. 

(1) The VTDM will use the time message from the tape to locate 
messages in the data base that match the target time. 

(2) When a message is located with the target time it will be tested to 
determine if it includes the selected scanner id. 

(3) If it does not include the selected scanner id it will be discarded. 

(4) If the time and date are correct, and the selected scanner id 
matches the scanner id in the message, the message will be 
formatted and written to the display data block. 

j) This system would provide visual documentation of the guard tour. 

(1) The visual documentation assures that the tour was performed. 

(2) Insurance companies would find this level of documentation 
superior to data that is only uploaded from an external device. 



C. Computer terminal data input applications. 

1. There are numerous instances where people enter data into terminals. 
The data entered is used in financial transactions. Credit systems and 
bank systems are two examples. 




Page seven of Exhibit B 




2. The terminals are part of larger mainframe or networked systems with 
numerous terminals. 



3. The records created with these systems frequently have time and date 
stamp fields. 



4. A Video Transaction Data Multiplexing can be used in these situations 
if the clock on the network or mainframe can be used to set and 
update the time on the VTDM computer. 



D. Convenience stores. 



1. In the store. 



a) The cash registers in the modern convenience store is a computer system comprised of 
several input terminals. The input terminals include: 

(1) Gas pump control. 

(2) Money order machine. 

(3) Cash register. 

(4) Lottery. 

(5) Credit card validator. 

b) In these stores, transactions from these terminals create time and date stamped records 
which are stored in the system. 

c) These records are used to produce summaries and reports for the chain. 

d) A file consisting of these time stamped records could be used as data for a VTDM 
application. 



2. At the gas pump. 
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a) Many convenience stores have gas pumps that employ card readers as the medium of 
payment. 

b) These pumps are on line to the validation center. 

c) As a worst case, if the time and date used on the data encoded on the VCR tape is set 
and updated from the validation center, the data records from the data center could be 
used as input for the data block when a tape is replayed. 



E. ATM machines. 



1. In the case of ATMs there is now a requirement that all transactions be 
video taped and that the time on the video tape be synchronized by the 
time on the users receipt and the mainframe transaction data base. 



2. With the new asynchronous VTDM we could install our proprietary 
single board AM90 micro controller to interface with the ATM card 
reader, and the ATM controller. 



a) The AM90 would input the card insert, card data stream, card remove signals ( See 
Drawing of simple encoder ). 

b) The AM90 would write this information along with ATM id number and time and 
date every 100 milliseconds. 

c) The transaction data from all of the ATM units in a single system are stored on the 
system mainframe. 

(1) These records include time, date, location along with card number, 
transaction type, etc. 

(2) These records can be output to a decoding replay station via serial 
communication or by means of transportable media such as floppy 
disk or tape. 
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(3) Tapes from the ATM sights could be replayed on the decoder replay 
station. 

(4) The time, date, ATM id could be used to index the correct data 
record which would then be displayed on the composite video 
picture. 

3. The VTDM ATM application has the following advantages: 

a) The data is placed on the picture at the time of replay. 

(1) Assures highest resolution for data characters. 

(2) Assures that data is placed where it will not interfere with 
important items in the picture that are important. 

b) The AM90 can be configured to input several ATM data streams while outputting to 
one or more VCRs. 

(1) This allows the user to employ multiple VHS VCRs, Quads or Video 
Time Division Multiplexers. 

(2) This can cut costs where there are several ATMs at a single sight. 



V. This new application uses techniques that store the transaction information and 
behavioral information separately. 



A. It differs from our first patent in that : 



1. The behavioral data ( the full motion video pictures ) and the 
transaction data ( time, date, lane, plaza, class, treadle count, error 
messages, transaction descriptor etc. ) are stored on separate media. 
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2. The behavioral data and the transaction data are not stored 
synchronously or necessarily simultaneously. 

a) The transaction data is put into a file on the computer disk. 

(1) The transaction data could come from an independent system, a 
time and attendance recording system, a data entry terminal, etc. 

(2) If the data is stored on removable media, the transaction data and 
the behavioral data could be sent to an audit center separately. 

b) The video pictures are stored on the VHS video tape recorder. 

3. Timing messages are stored on the video tape. 

a) These messages can be stored on the sound track, in the invisible portion of the video, 
or even in the visible video picture. 

b) We are using timing messages for every 100 milliseconds, the could come at a much 
faster pace if the application required. 

4. The transaction data and the behavioral data a combined on replay. 
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